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Summary 


Recent studies show that enrolling in algebra I in grade 8 works well for some students but 
backfires for others. The consequences of misplacement are most pronounced for students 
with weaknesses in key foundational areas that support algebra readiness, which frequently 
translates into difficulty reaching proficiency in higher level math in high school (Finkeh 
stein et al, 2012). 

One study of California students concludes that placing all grade 8 students in algebra I, 
regardless of their preparation, sets up many students to fail (Williams et al., 2011). Recent 
longitudinal analysis of California statewide assessment data suggests that students who 
do not take algebra I in grade 8 but are successful in general math have a better chance of 
succeeding in algebra I when they wait until grade 9 (Liang, Heckman, & Abedi, 2012). 

Are there effective ways to identify which students will be most likely to succeed (achieve 
proficiency) in algebra I in grade 8? To answer this question, Regional Educational Labo- 
ratory West, in collaboration with the eight members of the Silicon Valley Research Alli- 
ance (SVRA) districts, estimated the relationships between two assessments available to 
the SVRA districts and students’ likelihood of achieving proficiency in algebra I. These 
estimates were then used to determine how well different assessments identified students 
who were more likely to achieve proficiency in algebra I in grade 8. 

Key findings are: 

• Many SVRA districts use proficiency on the grade 6 math California Standards 
Test (CST) as the primary criterion for deciding whether to place students in 
algebra I in grade 8. 

° Students who score exactly at the proficiency cutpoint in grade 6 have less 
than a 40 percent chance of achieving proficiency in algebra I. 
o Students who score at least 17 points above the proficiency cutpoint (0.27 
standard deviation) in grade 6 have a more than 50 percent chance of achiev- 
ing proficiency in algebra I. 

• Other methods can boost the accuracy of algebra placement decisions in grade 8 
to 75 percent or higher: 

° Using the cutpoint on the grade 6 math CST associated with at least a 
50 percent chance of algebra I success instead of the cutpoint for proficiency 
status raises the accuracy of placement decisions from 69 percent to 75 percent. 
° Prediction accuracy reaches 77 percent using student results on a grade 7 test 
of algebra readiness developed as part of the newly available Mathematics 
Diagnostic Testing Project (MDTP). Students who achieve mastery in five 
or more MDTP topics have a 75 percent chance of achieving proficiency in 
algebra I. 

° Prediction accuracy reaches 78 percent using grade 7 CST scale scores. 
However, grade 7 scale scores are typically not available until after initial 
algebra I placements are made. 

• Predictions based on grade 7 MDTP results are consistent with those based on the 
grade 7 math CST more than 80 percent of the time. 

• Among students with discrepant predictions on the MDTP test and the CST, 
algebra I proficiency rates drop from about 78 percent to about 50 percent. In other 
words, even if students perform well on the grade 7 math CST, if they do not also 


perform well on the MDTP test at the end of grade 7, they have no more than a 
50-50 chance of achieving proficiency in algebra I. The same is true for students 
who perform well on the MDTP test but not on the grade 7 math CST. 

The findings suggest that the MDTP test makes a valuable contribution to decisions about 
algebra I placement. The MDTP results are largely consistent with those of the grade 7 
math CST, and they are available several months earlier than the CST results, which are 
often not available until the summer or fall of the grade 8 school year. Moreover, even 
after controlling for grade 6 math CST performance, these results indicate that the MDTP 
test identifies a set of measurable skills that predict algebra I proficiency. Practitioners may 
want to consider using MDTP results to aid in algebra I placement decisions and to identi- 
fy areas for focused support aimed at helping students succeed in algebra I. 
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Why this study? 


Math course placement decisions matter. Identifying students in grades 6 and 7 who are 
likely to succeed in algebra I in grade 8 can improve the ability of schools and districts to 
make appropriate placement decisions and provide appropriate instructional supports. In 
turn, this can help improve student success rates in algebra I. 

The importance of algebra I placement and proficiency rates 

Middle school math placement and progression are topics that are part of an active policy 
and practice discussion in California and elsewhere. Beginning in the 2008/09 school year, 
California’s State Board of Education recommended that students complete algebra I by 
the end of grade 8. Recent studies show that this strategy works well for some students but 
backfires for others. 

Between 2003 and 2009 the proportion of grade 8 students taking algebra I rose from 
32 percent to 54 percent (Williams et al., 2011). This increase resulted in a larger percent- 
age of grade 8 students scoring “proficient” or “advanced” (achieving proficiency) on the 
algebra I California Standards Test (CST; see box 1 for definitions of key terms used in 
this report) and in a larger number of grade 8 students scoring “far below basic” or “below 
basic” on the test (Williams et al, 2011). Williams et al. (2011) conclude that placing all 
grade 8 students in algebra I, regardless of their preparation, sets up many students to fail. 

The consequences of misplacement are most pronounced for students whose weaknesses 
in key foundational areas that support algebra readiness translate into difficulty reaching 
proficiency in higher level math in high school (Finkelstein et al., 2012). A 2012 Califor- 
nia study based on data for 24 districts found that most students who score proficient or 
advanced on standardized state tests in algebra I, geometry, and algebra II by grade 10 had 
participated in an accelerated math track beginning in middle school (Finkelstein et al., 
2012). However, that accelerated path is not available to students who do not demonstrate 
proficiency in grade 7 math. Another recent longitudinal analysis based on statewide 
assessment data revealed that students who fail the algebra I CST in grade 8 have a greater 
chance of repeating the course and failing the assessment again in grade 9 than their peers 
who pass the general math CST in grade 8 and then take algebra I in grade 9 for the first 
time (Fiang et al., 2012). 

The discussion of students’ preparation for and placement in algebra I is part of a much 
larger conversation across the United States about the progression of math instruction 
from early grades through high school. California is one of 45 states that recently adopted 
the Common Core State Standards in math, which were developed, in part, to address the 
common criticism that math education in the United States is “a mile wide and an inch 
deep.” The new standards “significantly narrow the scope of content and deepen how time 
and energy are spent in the math classroom ... so students gain strong foundations; [and 
the standards] carefully connect the learning within and across grades so that students 
can build new understanding onto foundations built in previous year” (Cocuzza, 2012). 
The standards are also intended to increase math proficiency for graduating high school 
students and to decrease the need for costly remediation in postsecondary education, 
which often results in a failure to complete the postsecondary degree (Venezia, Bracco, & 
Nodine, 2010; Complete College America, 2012). 
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Box 1. Definition of terms used in this report 


Although some terms may have more general meanings, the following terms are used in this 
report with these specific definitions. 

Accuracy. The percentage of students placed in algebra I in grade 8 who, based on their prior 
assessment scores, meet or exceed the probability cutoff for determining whether a student is 
predicted to achieve proficiency (score proficient or advanced) on the algebra I California Stan- 
dards Test (CST) in grade 8 and who actually achieve proficiency. Formally, it is the number of 
students who are predicted to achieve proficiency on the algebra I CST in grade 8 and actually 
do so, divided by the total number of students who are predicted to achieve proficiency on the 
algebra I CST in grade 8. This percentage is also called the “positive predictive value.” 

Algebra ready. Students who, based on their prior assessment scores, exceed the probability 
cutoff and who are predicted to achieve proficiency on the algebra I CST in grade 8. 

Algebra success. Students who actually achieve proficiency on the algebra I CST in grade 8. 

Correct classification. Whether students’ predicted outcomes (based on a probability cutpoint) 
from a logistic regression match students’ actual outcomes. In other words, whether students 
who are predicted to achieve proficiency on the algebra I CST in grade 8 actually do so and 
whether students who are predicted not to achieve proficiency actually do not. 

CST score. Student scale scores on the CST content test are classified into one of five levels: 
far below basic, below basic, basic, proficient, or advanced. Only students scoring at the profi- 
cient or advanced level are considered to have achieved proficiency in the content area being 
tested. Table A7 in appendix A lists the cutpoints for the grade 6 math, grade 7 math, and 
algebra I CSTs. 

Odds ratio estimate. The estimated change (increase or decrease) in odds of achieving profi- 
ciency on the algebra I CST in grade 8 associated with a unit increase in the predictor variable, 
as estimated in a logistic regression. An odds ratio of more than 1 indicates that the odds 
of achieving proficiency increase with a one-unit increase in the predictor variable. An odds 
ratio of less than 1 indicates that the odds of achieving proficiency decrease with a one-unit 
increase in the predictor variable. An odds ratio of 1 indicates that the odds of achieving pro- 
ficiency remain the same with a one-unit increase in the predictor. For example, for a model 
using grade 6 math CST scale scores as the predictor, an odds ratio estimate of 1.03 means 
that achieving proficiency on the algebra I CST in grade 8 is 1.03 times as likely for each one- 
point increase in the grade 6 math CST scale score. That is, the odds increase by 3 percent 
with each one-point increase in the score. 

Probability cutoff. The probability cutoff is a threshold probability used to determine whether 
a student is predicted to achieve proficiency on the algebra I CST in grade 8. The probability 
ranges from 0 to 1. If the predicted probability is equal to or higher than the cutoff, the student 
is predicted to achieve proficiency. If the predicted value is lower than the cutoff, the student is 
predicted not to achieve proficiency. This study applies various probability cutoffs from 0.5 to 
0.9, in increments of 0.1. 
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As school districts across the country move toward full implementation of the Common 
Core State Standards in math, districts will be examining course sequencing and the 
placement of students in existing and new courses. For example, how pre-algebra content 
is distributed across the years leading up to middle-school algebra will trigger discussions 
about which students should be following specific content sequences and at what pace. If 
the Common Core unfolds as envisioned, the result could be more students developing 
strong foundations in math concepts in the middle school years, leading to greater success 
in high school and beyond. 

Measures used to guide decisions on math course placement 

Increasing the number of students taking and achieving proficiency in algebra I in grade 8 
requires identifying students who are ready for algebra. Previous research on how schools 
and districts in California make placement decisions suggests an increasing reliance on 
test scores. 

The criteria used to determine math placement and the weights assigned to each criterion 
vary across the country and across school districts. Most districts rely on teacher recoin- 
mendations and course grades (Bitter & O’Day, 2010), with standardized math test scores, 
student/parent preferences, and counselor recommendations also factoring into the deci- 
sion (Hallinan, 2003). “Increasingly, school systems do not use fixed criteria to assign stu- 
dents to particular course levels. Teacher and counselor track-placement recommendations 
include, in addition to test scores and grades, highly subjective judgments about students’ 
personalities, behavior and motivation” (Oakes, Muir, & Joseph, 2000, p. 16). 

Over the last decade, school districts in California have been using multiple data sources 
to evaluate student preparation in grades 6 and 7 for algebra I in grade 8 (Williams et al, 
2011). These sources include scores on the math CST and on other assessments, as well as 
students’ grades in previous math courses and teacher recommendations. 

Despite the use of multiple measures in California, research suggests that CST scores have 
begun to figure more prominently in course placement decisions. Teachers report that CST 
scores and students’ prior academic performance were the most common considerations for 
placement decisions for grade 7 and grade 8 general math and algebra I courses (Williams 
et al., 2011). Evidence supports this approach, suggesting that CST scores are effective 
indicators of likely success in future math courses (Kriegler & Lee, 2006; Anderson & 
Newell, 2008). 
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What the study examined 


Regional Educational Laboratory West, in collaboration with the eight members of the 
Silicon Valley Research Alliance (SVRA) districts (Santa Clara County, California), 
looked at whether there are effective ways to identify which students will be most likely 
to achieve proficiency in algebra I in grade 8. To do that, they first looked at the place- 
ment strategies implemented in the SVRA districts and then estimated the relationships 
between the various assessments employed and students’ chances of achieving proficiency 
in algebra I. These estimates were then used to determine how well different assessments 
identified students who were more likely to achieve proficiency. 
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Algebra I placement strategies implemented in the Silicon Valley Research Alliance 


The participating SVRA districts 1 use multiple criteria for course placement decisions. In 
particular, feeder school districts to the East Side Union High School District have agreed 
to place students in algebra I based on the strategies presented in a 2010 memorandum of 
understanding that supports the use of multiple criteria (including course performance, 
teacher recommendations, diagnostic tests, and CST performance) and indicates that stu- 
dents who achieve proficiency on the grade 6 or grade 7 math CST should be placed in 
algebra I in grade 8. Despite the memorandum, there appears to be a great deal of variation 
within and across the districts in the criteria used for algebra I placement — though many 
districts continue to emphasize CST performance. 

This process is further complicated by the availability of results from new diagnostic 
assessments of grade 7 students’ math performance (and perhaps algebra readiness) before 
grade 7 math CST results are available. In the past, the SVRA districts relied on students’ 
prior math CST scores in making algebra I placement decisions. In 2010 some SVRA dis- 
tricts began to use the tests developed under the Mathematics Diagnostic Testing Project 
(MDTP), a joint project of the California State University and University of California 
higher education systems, to assist with these decisions. 

The MDTP offers cognitive diagnostic tests designed to provide students and teachers 
with information about student readiness for a broad range of math courses. This infon 
mation can help students identify areas where additional study or review is needed and 
help teachers identify topics and skills that need more attention in courses. Three types 
of MDTP results are available: the number of MDTP topics mastered, the level of mastery 
of each topic, and the topic scores. The MDTP tests can be administered online, and 
the results are available immediately after completion. CST results, by contrast, are not 
available until August, and in some years are not available until after the next school 
year begins. Therefore, some districts are exploring the use of MDTP results to assist with 
initial placement decisions. 

However, exactly which combination of tests would ensure the highest number of accu- 
rate placements remains unclear. The SVRA districts are interested in knowing whether 
and how to incorporate these different assessments into their math placement decisions 
in grade 8 and how different approaches would affect their students’ algebra I proficiency 
rates in grade 8. To address the districts’ interests, this study focuses on the relationships 
between students’ performance on these different assessments and their proficiency on the 
algebra I CST. 

The primary placement challenge is whether to assign students who have completed 
grade 7 math to a regular algebra I course or to a slower paced algebra/general math course 
in grade 8. 2 The grade 8 course placement process begins in the spring, as students are 
completing grade 7 (figure 1). Despite the existence of the memorandum of understanding, 
the factors applied and how they are incorporated in placement decisions vary. However, 
while math instructional teams rely on a variety of data, including course grades, teacher 
recommendations, and other subjective factors, in most cases assessment data factor 
heavily into the decision. 
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Figure 1. Timing of math assessments and placements in Silicon Valley Research 
Alliance districts 
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Placement decision for algebra I or slower paced math course in grade 8 is 
made in late spring through early fall: 

Initial decision made in late spring and summer using grade 6 math CST 
results, grade 7 MDTP results, or both. 

Final decision made primarily in August (and if needed in the early fall) 

using grade 7 math CST results to refine decision. 


CST is the California Standards Test. MDTP is Mathematics Diagnostic Testing Project. 
Source: Authors' construction based on information shared by the districts. 


Math instructional teams and counselors in the SVRA districts have access to three 
sources of formal assessments of math performance in making these decisions: 

• The grade 6 math CST, administered in the spring, with results generally available 
in August, around the time students begin grade 7. 

• The grade 7 MDTP test (also called the Algebra Readiness test), administered 
in the spring, a few weeks before the CST administration, with results available 
immediately. 

• The grade 7 math CST, administered in the spring, with results generally available 
in August, around the time students begin grade 8. 


Initial placement decisions, which have to occur before August, are made on the basis 
of grade 6 math CST scores. 3 If students achieve proficiency on the grade 6 math CST, 
they are generally placed in algebra I in grade 8. If results are available in time, districts 
sometimes use grade 7 math CST scores to refine the placement decisions right before the 
school year begins. In particular, if a student was placed into a class below algebra I but 
achieves proficiency on the grade 7 math CST, the student may be given an opportunity 
to enroll in algebra I at the beginning of the year. This process typically occurs after the 
master schedule has been set and students have received their class schedules. Students 
sometimes do not learn that they have the opportunity to attend a more advanced math 
class until they arrive in their initially assigned math class on the first day of school. 


Research questions 

This report addresses two broad research questions: 

1. What is the relationship between proficiency on the algebra I CST in grade 8 and 
scores on the grade 6 math CST, the grade 7 MDTP test, and the grade 7 math CST? 

2. How can test scores better assist with initial placement decisions? 
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With MDTP scores now available in the spring of students’ grade 7 year, the SVRA dis- 
tricts were interested in learning whether and how to use these data to improve the place- 
ment process. Specifically, the districts wanted to understand how well the MDTP test 
measures students’ content knowledge on the grade 7 math concepts that are important 
for proficiency in algebra I and whether MDTP scores are more accurate than grade 6 
math CST scores in placing students in algebra I in grade 8. The districts also wanted to 
know whether using scores on both the grade 6 math CST and the grade 7 MDTP test 
might improve placement decisions compared with using scores on either test alone. And, 
since placement decisions are sometimes revisited once grade 7 CST scores are available, 
the SVRA districts are interested in whether placement decisions based on scores on the 
grade 6 math CST and the grade 7 MDTP tests have a different success rate than deci- 
sions based on scores on the grade 7 math CST alone. 

The study used grade 6 and 7 math CST scores, grade 7 MDTP test scores, and algebra I 
CST scores for students enrolled in algebra I in grade 8 in the 2011/12 school year in 
SVRA districts (2,579 students from five districts; see table A1 in appendix A). The study 
used cross-tabulation and logistic regression analyses to study the association between stu- 
dents’ proficiency on the algebra I CST in grade 8 and their prior performance on the 
grade 6 math CST, grade 7 MDTP test, and grade 7 math CST. In particular, the analysis 
examined how algebra I proficiency rates in grade 8 varied with grade 6 math CST profi- 
ciency (see details of the analysis in appendix A). 

Next, the percentage of students achieving proficiency on the algebra I CST in grade 8 
was calculated for each grade 6 math CST scale score to identify the grade 6 math CST 
scale score associated with students having at least a 50 percent probability of achieving 
proficiency on the algebra I CST in grade 8. The predictive accuracy of this cutpoint score 
was then compared with that of the CST cutpoint for proficiency status. Logistic regres- 
sion was used to compare the accuracy of predictions based on different scores, including 
grade 6 math CST scale scores, grade 7 MDTP scores, and grade 7 math CST scale scores. 
These results were used to examine the implications of algebra I placement decisions in 
grade 8 made using different sources of information. Appendix A describes the analysis 
methods in detail. Appendix B summarizes the logistic regression models used in the 
study, specifies which predictors were included in each model, and provides information 
about how each model performed in predicting algebra I success in grade 8. 
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The evidence presented in this report could help the SVRA districts better understand the 
indicators that predict algebra I success, refine their placement criteria, streamline their 
placement process, and increase the number of algebra-ready students placed in algebra I 
in grade 8. The findings from this study could also inform discussions and further analysis 
of success indicators and placement criteria for math courses in other districts in Califor- 
nia as well as the rest of the nation. 


Findings 


Study findings are reported on the relationship between algebra I proficiency in grade 8 
and scores on the grade 6 math CST, the grade 7 MDTP test, and the grade 7 math CST 
and on how accurately the different assessments predict proficiency among students placed 
in algebra I in grade 8. 
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Placing students in algebra I in grade 8 based on achieving proficiency on the grade 6 math CST 
results in an accurate placement for 69 percent of students 

Most SVRA districts place students in algebra I in grade 8 if they score at or above the 
proficiency threshold on the grade 6 math CST. To examine the accuracy, or positive pre- 
dictive value, of this placement measure, the study calculated the percentage of students 
who achieved proficiency on the grade 6 math CST who also achieved proficiency on the 
algebra I CST in grade 8. 

Among the 1,451 students who achieved proficiency on the grade 6 math CST, 69 percent 
(996) also achieved proficiency on the algebra I CST in grade 8 (table 1). The remaining 
31 percent (455) of the students identified as algebra ready based on the grade 6 math CST 
failed to achieve proficiency. 

Using students’ grade 6 math CST scale score rather than proficiency status increases the accuracy 
of algebra I placement decisions in grade 8 from 69 percent to 75 percent 

Moving beyond the simple zero or one indicator of proficiency status (achieving proficiency 
or not), the study used continuous grade 6 math CST scale scores and a logistic regression 
(see appendix A) to predict student probability of achieving proficiency on the algebra I 
CST in grade 8. Each student was assigned a predicted probability based on the grade 6 
math CST scores. Students who had a predicted probability of 50 percent or higher were 
considered algebra ready. 

Compared with decisions based on proficiency alone, this approach increased the accuracy 
of algebra I placement decisions in grade 8 by 6 percentage points, suggesting that it is a 
more accurate way to predict proficiency on the algebra I CST in grade 8. Some 75 percent 
of students who had been classified as algebra ready based on their grade 6 math CST scale 
score achieved proficiency on the algebra I CST in grade 8 (857 of a total of 1,145 students; 
table 2), compared with 69 percent classified as algebra ready based on their grade 6 math 
CST proficiency status alone (see table 1). 

To have more than a 50 percent chance of achieving proficiency on the algebra I CST in grade 8, 
students need to score at least 17 points above the proficiency cutpoint on the grade 6 math CST 

A predicted probability of achieving proficiency on the algebra I CST in grade 8 was cal- 
culated for each grade 6 math CST score (figure 2). Simply scoring at the proficient level 
(a scale score of 350) is not enough to give a student a greater than 50 percent chance 
of achieving proficiency on the algebra I CST in grade 8. Students who scored 350 had 


Among students 
who took algebra I 
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Table 1. Some 69 percent of students who achieved proficiency on the grade 6 
math CST also achieved proficiency on the algebra I CST in grade 8 (percent) 


Status on grade 6 math CST 

Status on algebra 1 
Proficient 

1 CST in grade 8 
Not proficient 

Proficient (n = 1,451) 

69 

31 

Not proficient (n = 1,128) 

20 

80 

CST is the California Standards Test. 

Source: Authors' analysis of primary data collected for the study; see appendix A. 
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Table 2. Some 75 percent of students predicted to be proficient on the algebra I 
CST in grade 8 based on their grade 6 math CST scale score actually achieved 
proficiency (percent) 


Predicted status on algebra 1 CST in grade 8 

Actual status on algebra 1 
CST in grade 8 

based on grade 6 math CST scale score 

Proficient 

Not proficient 

Proficient (n = 1,145) 

75 

25 

Not proficient (n = 1,434) 

26 

74 


CST is California Standards Test. 

Note: A probability cutoff of 0.5 was used. 

Source: Authors' analysis of primary data collected for the study; see appendix A. 


Figure 2. Students who scored at the proficient level on the grade 6 math CST had 
less than a 40 percent chance of achieving proficiency on the algebra I CST in grade 8 


Predicted probability of proficiency on algebra I CST in grade 8 



Source: Authors' analysis of primary data collected for the study; see appendix A. 


only a 39 percent probability of achieving proficiency. Thus, classifying students as algebra 
ready based on proficiency on the grade 6 CST ends up including some students who have 
less than a 40 percent chance of achieving proficiency on the algebra I CST in grade 8. 
Raising the probability to 50 percent required a scale score of 367 or higher on the grade 6 
math CST (0.27 standard deviation above the proficiency threshold). 

Depending on the assessment used to predict algebra I proficiency in grade 8, prediction accuracy 
ranges from 75 percent to 78 percent 

To assess the potential contribution of the MDTP test to accurate placement decisions, 
several combinations of grade 6 and 7 assessments were used to predict algebra I proficien' 
cy in grade 8. The combinations were chosen to assess the options most readily available 
to the SVRA districts for adding the MDTP test to the placement process, including using 
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Table 3. The accuracy of the assessment measures used for predicting proficiency 
on the algebra 1 CST in grade 8 ranges from 75 percent to 78 percent (percent) 

1 Model 

Predictor 

Accuracy j 

i 

Grade 6 math CST scale score 

75 

2 

Grade 7 MDTP (number of topics mastered) 

77 

3 

Grade 7 MDTP (seven topic scores) 

76 

4 

Grade 6 math CST scale score and grade 7 MDTP (seven topic scores) 

77 

5 

Grade 7 math CST scale score 

78 


CST is California Standards Test. MDTP is the Mathematics Diagnostic Testing Project. 

Note: A probability cutoff of 0.5 was used because it typically yields the highest percentage of correctly classi- 
fied students (see table B2 in appendix B). The accuracy based on other probability cutoffs is reported in table 
B3 in appendix B. 

Source: Authors' analysis of primary data collected for the study; see appendix A. 


the MDTP test alone, using it in combination with the grade 6 math CST results, and 
using the MDTP topic scores or using MDTP mastery indicators. 

The model including both grade 6 math CST scale score and grade 7 MDTP topic scores 
(model 4, deemed to be the best-fitting model as shown in table B1 in appendix B) resulted 
in accurate predictions of success 77 percent of the time, compared with 75 percent accu- 
racy for the model based on grade 6 math CST scale score alone (model 1), 77 percent for 
the model based on the number of MDTP topics mastered (model 2), 76 percent for the 
model based on seven MDTP topic scores (model 3), and 78 percent for the model based 
on grade 7 math CST scale score alone (model 5; table 3). Other than the low success rate 
of using grade 6 math CST proficiency alone for algebra I placement decisions in grade 8, 
these findings suggest few differences in assignment decisions and proficiency rates among 
the other assessment options. 


Other than the low 
success rate of 
using grade 6 math 
CST proficiency 
alone for algebra I 
placement 
decisions in 
grade 8, the 
findings suggest 
few differences 
in assignment 
decisions and 
proficiency rates 
among the other 
assessment options 


The findings also indicate that relying solely on reports of the number of topics mastered 
on the grade 7 MDTP, which are available in the spring preceding grade 8 and require no 
matching of student records across years, results in an accuracy rate that is within a single 
percentage point of the success rates generated using either the grade 7 math CST or a 
combination of grade 6 math CST and grade 7 MDTP topic scores. 


Using grade 7 math CST scores to fine-tune placement decisions before the beginning of grade 8 
does not necessarily result in more accurate placement outcomes 

Schools and districts often revisit spring placement decisions once grade 7 math CST 
scores are available, generally around August. They use these results to place students who 
achieve proficiency on the grade 7 math CST in algebra I in grade 8 even if their grade 6 
math CST score was not high enough to merit algebra I placement. To examine the poten- 
tial effects of such a policy, this study compares the consistency of placement decisions 
based on the data available at the end of grade 7 (grade 6 math CST scale scores and 
grade 7 MDTP topic scores) with the consistency of placement decisions based on grade 7 
math CST scale scores (not available until the summer). To the extent that placement 
decisions based on the combination of grade 6 CST scale scores or grade 7 MDTP topic 
scores align with the predictions based on grade 7 CST scale scores, revised placements 
might be considered unnecessary. To the extent that students placed in algebra I in grade 8 
based on the addition of the grade 7 math CST scale scores are less likely to achieve profi- 
ciency, using those data might even be contraindicated. 
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In general, there is a high degree of overlap in the predictions based on the combination 
of grade 6 math CST scale scores and grade 7 MDTP topic scores and predictions based 
on grade 7 math CST scale scores alone. Of students predicted to achieve proficiency in 
algebra I in grade 8 based on the combination of grade 6 math CST scale score and grade 7 
MDTP topic scores, 81 percent (970 of a total of 1,201 students) are also predicted to do so 
based on their grade 7 math CST scale score alone (table 4). Similarly, 83 percent of the 
students predicted not to achieve proficiency based on their grade 7 math CST scale score 
are also predicted not to do so based on the combination of their grade 6 math CST scale 
score and grade 7 MDTP topic scores. 


Further, 52 percent of the students who were predicted not to achieve proficiency on the 
algebra I CST in grade 8 based on the combination of grade 6 math CST scale score and 
grade 7 MDTP topic scores but were predicted to do so based on grade 7 math CST scale 
score alone did achieve proficiency (table 5). Similarly, 51 percent of students who were 
predicted not to achieve proficiency based on grade 7 math CST scale score but were pre- 
dieted to do so based on the grade 6 math CST scale score and grade 7 MDTP topic scores 
did achieve proficiency. 


Table 4. Of students predicted to achieve proficiency in algebra I in grade 8 based 
on the combination of grade 6 math CST scale score and MDTP topic scores, 

81 percent are also predicted to do so based on their grade 7 math CST scale 
score alone 


Predicted status on algebra 1 CST in 

Predicted status on algebra 1 CST in grade 8 based 

grade 8, based on the combination 

on grade 7 math CST scale score alone (model 5) 

of grade 6 math CST scale score and 
grade 7 MDTP topic scores (model 4) 

Proficient 

Not proficient 

Proficient (n = 1,201) 

81 

19 

Not proficient ( n = 1,378) 

17 

83 


Relying on grade 7 
math CST scale 
scores in late 
August or early 
September to 
place students 
in algebra I in 
grade 8 is unlikely 
to increase 
the overall 
percentage of 
students achieving 
proficiency on 
the algebra I 
CST in grade 8 


CST is the California Standards Test. MDTP is the Mathematics Diagnostic Testing Project. 
Note: A probability cutoff of 0.5 was used. 

Source: Authors' analysis of primary data collected for the study. 


Table 5. Students with discrepant predictions had a 51- 
achieving proficiency on the algebra 1 CST in grade 8 

-52 percent chance of 

Prediction 

Total number of 
students with 
discrepant 
predictions 

Number of 
these students 
who achieved 
proficiency on 
algebra 1 CST 
in grade 8 

Percentage 
who achieved 
proficiency on 
algebra 1 CST 
in grade 8 

Grade 6 math CST scale score and grade 7 math 
MDTP topic scores = Not proficient 
Grade 7 math CST = Proficient 

230 

119 

52 

Grade 6 math CST scale score and grade 7 math 
MDTP topic scores = Proficient 
Grade 7 math CST = Not proficient 

231 

110 

51 


CST is the California Standards Test. MDTP is the Mathematics Diagnostic Testing Project. 
Source: Authors' analysis of primary data collected for the study. 
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These findings indicate that relying on grade 7 math CST scale scores in late August or 
early September to place students in algebra I in grade 8 is unlikely to increase the overall 
percentage of students achieving proficiency on the algebra I CST in grade 8. In fact, 
the findings suggest that adding students to algebra I courses in this manner (top row of 
table 5) could lower algebra I proficiency rates, since only 52 percent of students placed in 
this way would achieve proficiency. The findings also suggest that placing students in non- 
algebra math classes because grade 7 math CST scale scores were lower than the grade 6 
CST scale scores and grade 7 MDTP topic scores (bottom row) could reduce the number 
of students who enroll in algebra I in grade 8 and achieve proficiency on the algebra I CST 
in grade 8. 


MDTP scores in five of the seven topics are significant predictors of algebra I proficiency 

Scores on five of the seven MDTP topics are significant predictors of students’ odds of 
achieving proficiency on the algebra I CST in grade 8, even after controlling for grade 6 
CST scale score (table 6). The exceptions are data analysis, probability, and statistics and 
geometric measurement and coordinate geometry. The estimated odds ratios associated 
with the five significant topics are all larger than 1, indicating that a one percentage point 
increase in scores on any of those topics is associated with an increase in the odds of 
achieving proficiency. For example, holding other predictor scores constant, a one percent- 
age point increase on MDTP topic area 6, integers, is associated with a 26 percent increase 
in the odds of achieving proficiency on the algebra I CST in grade 8. 

This finding suggests that educators in the SVRA districts could consider emphasizing 
the skills identified by the five topic tests that are significant predictors of the likelihood of 
algebra I success in grade 8. The findings also suggest that the MDTP results could be used 
to identify individual student needs with respect to several dimensions of skills that are 
significant predictors of algebra I proficiency in grade 8. 


MDTP results 
could be used to 
identify individual 
student needs 
with respect to 
several dimensions 
of skills that 
are significant 
predictors 
of algebra I 
proficiency 
in grade 8 


Table 6. When including both grade 6 math CST scale scores and grade 7 MDTP 
topic scores as predictors, five of the seven MDTP topics were significant 
predictors of algebra I success in grade 8 


Predictor 

Odds ratio 

Standard error 9 

Grade 6 math CST scale score 

1.02** 

0.003 

MDTP topic 

1 data analysis, probability, and statistics 

1.00 

0.062 

2 decimals, their operations and applications; percent 

1.12** 

0.036 

3 exponents and square roots; scientific notation 

±,22** 

0.048 

4 fractions and their applications 

1.19** 

0.094 

5 geometric measurement and coordinate geometry 

1.02 

0.060 

6 integers 

1.26** 

0.035 

7 literals and equations 

1.18** 

0.052 


CST is the California Standards Test. MDTP is the Mathematics Diagnostic Testing Project, 
♦♦significant at p<.05. 

a. Adjusted for taking into account the nested nature of the data (students are nested in districts). 
Source: Authors' analysis of primary data collected for the study; see appendix A. 
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Table 7. Student proficiency on the algebra 1 CST in grade 8 is linearly correlated 
with the number of MDTP topics mastered 

Number of topics mastered on the grade 7 MDTP 

Proficient on the algebra 1 CST in grade 8 
(percent) 

0 


11 

1 


23 

2 


35 

3 


52 

4 


68 

5 


75 

6 


85 

7 


94 

CST is the California Standards Test. MDTP is the Mathematics Diagnostic Testing Project. 
Source: Authors' analysis of primary data collected for the study. 


Students who master four or more topics on the grade 7 MDTP have a nearly 70 percent or higher 
chance of achieving proficiency on the algebra I CST in grade 8 

In addition to providing continuous measures of skills, the MDTP test indicates whether 
students have mastered the seven topics it assesses. The findings suggest a strong, posi- 
tive relationship between proficiency on the algebra I CST in grade 8 and the number of 
grade 7 MDTP topics students master in grade 7 (table 7). The more MDTP topics students 
master, the higher the probability they will achieve proficiency on the algebra I CST in 
grade 8. Students who master three MDTP topics have a 52 percent probability of achiev- 
ing proficiency. For students who master four topics, the probability rises to 68 percent, and 
for those who master five, it rises to 75 percent. This suggests that a threshold of mastering 
four MDTP topics would have a prediction accuracy close to that of grade 6 math CST 
proficiency and that a threshold of five MDTP topics would perform as well as a regression 
model based on grade 6 math CST scale scores. 


Districts might 
consider using the 
number of topics 
students master in 
the grade 7 MDTP 
to assist with 
course placement 
decisions. MDTP 
accuracy rates 
are the same as 
those generated 
by the CST 


This result reinforces the finding that the MDTP is an effective tool for identifying student 
progress on a set of specific skills that predict algebra I proficiency in grade 8. SVRA dis- 
tricts might consider using the number of topics students master in the grade 7 MDTP to 
assist with course placement decisions. MDTP accuracy rates are the same as those gener- 
ated by the CST. 


What the findings mean 

In addition to the questions on the relationship between different assessments and algebra I 
proficiency in grade 8, the study was motivated by an interest in understanding how test 
scores can be used to potentially improve algebra I placement decisions. 

How can test scores better assist with initial placement decisions? 

Increasing the number of students who pass algebra I by the end of grade 8 is an important 
policy objective in California and other states. As the number of students taking algebra I 
in grade 8 has increased, so has the number of students who fail to achieve proficiency 
on the algebra I CST in grade 8. The findings of this study have several implications for 
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policy and practice related to accurately identifying students who are likely to achieve pro- 
ficiency in algebra I in grade 8. 

By itself, proficiency on the grade 6 math CST is not an effective indicator of algebra 
readiness: Most students who score at the proficient level (a score of 350) do not achieve 
proficiency on the algebra I CST in grade 8. Students have to score at least 0.27 standard 
deviation above the proficiency cutpoint to have even a 50 percent probability of achiev- 
ing proficiency. 

This finding suggests that the accuracy of placement decisions could be improved by 
moving beyond proficiency on the CST and relying instead on CST scale scores to identify 
students who have at least a 50 percent chance of succeeding in algebra I. 

Moving grade 8 students into algebra I based on their grade 7 math CST scale score is 
unlikely to increase the overall percentage of students who achieve proficiency on the 
algebra I CST in grade 8. Combining grade 6 math CST scale score and grade 7 MDTP 
topic scores yields largely the same placement decisions as relying on grade 7 math CST 
scale score. And in cases when grade 7 math CST scale scores suggest a higher place- 
ment than do the combination of grade 6 math CST scale score and MDTP topic scores, 
the probability of student success in algebra I in grade 8 drops from 78 percent to about 
50 percent. The probability also drops to about 50 percent for students who perform well 
on the grade 6 math CST and grade 7 MDTP test but not on the grade 7 math CST. 

This suggests that even if placing students in algebra I in grade 8 based on grade 7 math 
CST scale score, after initial placements have been made based on the combination of 
grade 6 CST scale score and grade 7 MDTP topic scores, increases the number of students 
enrolled in algebra I in grade 8, it is unlikely to increase the percentage of students who 
achieve proficiency on the algebra I CST in grade 8. This result also suggests that moving 
students who perform poorly on the grade 7 math CST into a lower level math class could 
reduce the number of students who take algebra I in grade 8 but fail to achieve proficien- 
cy. So, since revising placements “up” based on grade 7 CST scale scores is unlikely to 
increase the number of students who achieve proficiency, districts and schools might want 
to consider revising placements “down” or identifying students for additional support based 
on their grade 7 math CST scale scores. 

Implications for education practitioners 

The MDTP test clearly identifies a set of measurable skills that predict algebra I proficiency 
and thus can make a timely and valuable contribution to algebra I placement and support 
decisions in grade 8. Education practitioners might consider using MDTP results to iden- 
tify topics for focused support. For example, SVRA districts could consider placing more 
emphasis on the skills that these analyses show are strongly related to algebra I proficiency. 
Districts could also consider providing additional support (such as instruction tailored to 
specific topics) to students based on the MDTP topics in which they fail to achieve mastery 
and using mastery on five MDTP topics as an instructional target, because 75 percent of 
the students who reach this degree of mastery achieve proficiency in algebra I in grade 8. 
For example, districts could conduct a content analysis of grade 5-7 math courses to 
determine how closely the content taught is aligned with the MDTP topics that predict 
algebra I proficiency. 


Moving students 
who perform poorly 
on the grade 7 
math CST into a 
lower level math 
class could reduce 
the number of 
students who 
take algebra I in 
grade 8 but fail to 
achieve proficiency 
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Staff in SVRA districts could consider relying primarily on MDTP results to make 
algebra I placement decisions in grade 8. The MDTP results are particularly useful for new 
students from other states who did not take the grade 6 or 7 math CST. MDTP results 
are as accurate at predicting algebra I proficiency as grade 6 math CST scale score. The 
MDTP results are readily available without complicated merging of data from the previous 
academic year, reflect performance in the most recent school year, and provide specific 
diagnostic information on each student’s strengths and weaknesses. Moreover, evidence 
suggests that revising algebra I placement decisions based on grade 7 math CST scale score 
does not necessarily increase the percentage of students who are likely to achieve profi- 
ciency on the algebra I CST in grade 8. 


Because many districts in California administer the optional MDTP tests in addition to 
the mandatory CSTs, those districts could also benefit from the results of this study. 

Finally, understanding the relationships between prior performance on various assessments 
and future achievement in algebra I in grade 8 can inform course placement decisions 
during and after the districts’ transition to the Common Core State Standards in math. 
Since the transition in assessment strategies at the state level includes the phasing out of 
the CSTs, the information provided by the MDTP test will provide districts with a consis- 
tent assessment tool for making placement decisions. 

The math CSTs and grade 7 MDTP test used in this study may not fully align with the 
Common Core math standards, but during and after the assessment transitions, school 
districts can make their placement decisions based on the study findings. In particular, the 
SVRA districts can use grade 7 MDTP topic scores to determine whether students have 
mastered specific topics as well as to make algebra I placement decisions in grade 8. In 
addition, MDTP results could help districts identify needed instructional support consis- 
tent with implementation of the algebra I Common Core State Standards. Once enough 
student assessment data related to the Common Core State Standards are available for 
secondary analyses, those data along with the revised MDTP results 4 can be used to adapt 
and revise the current study to assist districts with their placement decisions. 


The MDTP test 
clearly identifies a 
set of measurable 
skills that 
predict algebra I 
proficiency and 
thus can make a 
timely and valuable 
contribution 
to algebra I 
placement and 
support decisions 
in grade 8 


Study limitations 


The study was limited to students from the eight SVRA districts. Only the grade 8 cohort 
for 2011/12 had the data needed for analysis. The study did not examine the proficien- 
cy rates of students who were not placed in algebra I in grade 8. Because most of the 
SVRA districts emphasize use of assessments for placing students in math courses, the 
study focused on the test data used to assist with placement decisions rather than on other 
factors, such as course performance and teacher recommendations. 
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Appendix A. Detailed study design 


This appendix presents detailed information on the study design, including the study 
sample, data, methods, and limitations. 

Analytic sample 

The analytic sample for this study consisted of the students enrolled in algebra I in grade 8 
in Silicon Valley Research Alliance (SVRA) districts in the 2011/12 school year and for 
whom scores were available for all four of the tests of interest: grade 7 Mathematics Diag- 
nostic Testing Project (MDTP), grades 6 and 7 math California Standards Tests (CSTs), 
and algebra I CST. Students who were enrolled in an individualized education program 
(141 students) were omitted from the sample because these students may receive a modified 
curriculum. The analytic sample consisted of 2,579 students (table Al). 

Data sources and key measures 

This study relies on two types of assessment data: the grade 7 MDTP assessment and the 
grade 6-8 CST. 

The Mathematics Diagnostic Testing Project assessments. Eight SVRA districts have 
started to use MDTP tests, specifically, the Algebra Readiness test for grade 7 students. 
That test consists of 45 multiple-choice items in seven topics (table A2). For each topic, 
the MDTP has designated a “critical level,” which is the minimum number of correct 
responses required for a student to show adequate preparation in that topic. 

That information was used to create a set of dichotomous variables indicating whether 
students have mastered each area. These variables were also used to determine the number 
of grade 7 MDTP topics that students have mastered. 

MDTP provides reports at the class and individual levels. 5 At the class level teachers can 
use the itemdevel analysis across students within a class to adjust and improve instruction. 


Table Al. Study sample 




Number of 

Analytic sample 

District 

Total number 
of students 
enrolled in 
grade 8 
(2011/12) 

Number of 
students 
with all four 
test scores 

students 
with an 
individualized 
education 
program 

Number of 
students 

Percent of total 
enrollment 

Alum Rock 

1,273 

522 

9 

513 

40 

Berryessa 

933 

370 

16 

354 

38 

Franklin-McKinley 

956 

286 

9 

277 

29 

Milpitas 

738 

0 

0 

0 

0 

Mt. Pleasant 

291 

0 

0 

0 

0 

Oak Grove 

1,263 

572 

25 

547 

43 

San Jose 

2,394 

970 

82 

888 

37 

Sunnyvale 

593 

0 

0 

0 

0 

Total 

8,441 

2,720 

141 

2,579 

31 


Source: Authors' analysis of enrollment data from http://www.ed-data.kl2.ca.us (retrieved May 20, 2013). 
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Table A2. Number of items by topic, grade 7 MDTP Algebra Readiness test 


1 Topic 

Number of items 

Critical level n 

1 data analysis, probability, and statistics 

6 

4 

2 decimals, their operations and applications; percent 

8 

6 

3 exponents and square roots; scientific notation 

4 

3 

4 fractions and their applications 

8 

6 

5 geometric measurement and coordinate geometry 

6 

4 

6 integers 

6 

4 

7 literals and equations 

7 

5 

Total 

45 


MDTP is Mathematics Diagnostic Testing Project. 
Source: Provided by the MDTP test developer. 


Analysis of responses to the sample question in figure A1 shows how itemdevel analysis can 
inform teachers about their students’ progression in understanding math topics (table A3). 

Because only 17 of 98 students (17 percent) answered the question correctly, the question 
is considered a difficult one. If the distractors (choices that are not the correct answer) 
consisted of random numbers, there would be no direct way to understand why most stu- 
dents failed to answer the question correctly. If the distractors are designed to reflect some 
common misconceptions, students’ responses could provide information about where and 
why the difficulty occurred. In this example, choice C (incorrect) was the most popular 
(selected by 56 of 98 students). This outcome suggests the following: 

“...most of the students converted the denominator to l/(a - b ) before inverting 
and multiplying. It appears that they are still adding fractions by adding denomi- 
nators instead of first finding a common denominator. Another explanation may 
be that each term in the denominator was first inverted, giving a — b, and then the 
result inverted and multiplied to yield (C). In any case, an analysis of the students’ 
responses to this item would show that a large percentage of students are having 
difficulty with literal fractions” (MDTP website at http://mdtp.ucsd.edu/TestResults. 
shtml, retrieved May 31, 2012). 


Figure Al. A sample MDTP question 

a-b 


a b 

(A) -ab 

(B) ab 

(C) (a -by 

(D) a 2 + b 2 



MDTP is Mathematics Diagnostic Testing Project. 

Source: MDTP website at http://mdtp.ucscl.edu/TestResults.shtml (retrieved May 31, 2012). 
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Table A3. Marked-up item-level analysis for the sample MDTP question 

Response 

Number of students 

A a 

17 

B 

4 

C 

56 

D 

4 

E 

13 

Omitted 

4 

Total 

98 

MDTP is Mathematics Diagnostic Testing Project, 
a. Correct answer. 

Source: Data were made up by the authors. 



Based on this information, the teacher can spend more time helping students find the 
common denominator for (1/a - 1/b) and explaining that the subtraction in the denomina- 
tor needs to be done before the division or multiplication. The teacher could also demon- 
strate how to do this using numbers instead of letters. 

Individual-level reports provide each student’s raw score for each topic (figure A2). The 
critical level for each topic is also included to help teachers determine whether the student 
has mastered each topic. 

The sample individual report in figure A2 indicates that the student mastered two topics 
and needed improvement on the rest. Teachers can use this information to help place 
students in appropriate math courses in the subsequent semester and to identify interven- 
tions that target the topics students have not yet mastered. These data could be used more 
effectively for both placement decisions and instructional intervention if teachers and dis- 
tricts had more information on the relationship between algebra I success in grade 8 and 
mastery of these topics. 


Figure A2. Sample individual-level report from an MDTP test 


Your Score Critical Level Total Possible 

Congratulations! Your results indicate that you have done well in each of the following topics: 

Literals & Equations 3 3 4 

Data Analysis, Probability, & Statistics 5 4 6 

However, your results indicate you need review in the following topics: 

Integers 5 6 8 

Fractions and their Applications 4 5 7 

Decimals, their Operations & Applications; Percent 5 6 9 

Exponents and Square Roots; Scientific Notation 2 3 4 

Your results indicate you need substantial review in the following topics: 

Geometric Measurement and Coordinate Geometry 2 5 7 

Your total score is 26 out of 45, which is 58%. We hope you find this information helpful. Please contact 
your teacher for specific activities and assignments that will aid in any necessary review. 


MDTP is Mathematics Diagnostic Testing Project. 

Source: MDTP website at http://mdtp.ucsd.edu/pdf/StudentLetter.pdf (retrieved May 31, 2012). 
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California Standards Tests for math for students in grades 6 and 7 and for algebra I. As 
part of California’s accountability system, the math CST is given in the spring to monitor 
how well schools and students are performing. 6 Tables A4-A6 summarize the topics and 
the number of items associated with each topic for the grade 6-8 math CSTs. 

On each CST student performance is categorized into one of the following levels: far below 
basic, below basic, basic, proficient, or advanced. Table A7 lists the performancedevel cut- 
points for grade 6 math, grade 7 math, and algebra I CSTs. Only students performing at 
the proficient or advanced level are considered proficient in the content area being tested. 


Table A4. Topics included in the grade 6 math CST 

California content standard 

Number of items 

Percent of total 

Number sense 

25 

39 

Algebra and functions 

19 

29 

Measurement and geometry 

10 

15 

Statistics, data analysis, and probability 

11 

17 

Mathematical reasoning 

Embedded in other 
content standards 

na 

Total 

65 

100 

CST is California Standards Test; na is not applicable. 

Source: http://www.cde.ca.gov/ta/tg/sr/technicalrpts.asp (retrieved May 31, 2012). 


Table A5. Topics included in the grade 7 math CST 


California content standard 


Number of items 


Percent of total 


Number sense 22 34 

Algebra and functions 25 38 

Measurement and geometry 13 20 

Statistics, data analysis, and probability 5 8 

Mathematical reasoning Embedded in other na 

content standards 


Total 


65 


100 


CST is California Standards Test; na is not applicable. 

Source: http://www.cde.ca.gov/ta/tg/sr/technicalrpts.asp (retrieved May 31, 2012). 


Table A6. Topics included in the algebra I CST 


California content standard Number of items Percent of total 


Number properties, operations, and linear equations 17 26 

Graphing and systems of linear equations 14 22 

Quadratics and polynomials 21 32 

Functions and rational expressions 13 20 

Total 65 100 

CST is California Standards Test. 

Source: http://www.cde.ca.gov/ta/tg/sr/technicalrpts.asp (retrieved May 31, 2012). 
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Table A7. Performance-level cutpoints for grade 6 math, grade 7 math, and 
algebra I CSTs 


1 CST 

Below basic 

Basic 

Proficient 

Advanced ■ 

Grade 6 math 

253 

300 

350 

415 

Grade 7 math 

257 

300 

350 

414 

Algebra 1 

253 

300 

350 

428 


CST is California Standards Test. 

Note: Performance-level cutpoints remain the same from year to year. 
Source: California Department of Education, 2011. 


CST and MDTP content areas. The CSTs are summative assessments with a focus on 
students’ overall performance. The MDTP tests are diagnostic assessments, intended to 
guide efforts to improve student learning and teacher instructional practices. Depending 
on need, the MDTP tests can be administered before, during, or at the end of a semester. 
In general, the grade 7 MDTP test is given in the spring of grade 7 and used to assess 
student mastery of topics necessary for proficiency in algebra I. Despite the fundamental 
difference in test design and purpose, some of the topics covered in the grade 7 CST and 
MDTP test overlap (table A8). The approaches to reporting student performance differ 
in the two tests: The CST focuses on students’ overall performance, whereas the MDTP 
reports student learning by topic. 

Research questions 

This section describes the analytic strategies used to answer the following research 
questions: 

• To what extent did grade 6 math CST scores predict student proficiency on the 
algebra I CST in grade 8? 

• To what extent did grade 7 MDTP performance predict student proficiency on the 
algebra I CST in grade 8? 


Table A8. Topics assessed: grade 7 math CST and grade 7 MDTP test 

1 Topics/contents standards 

CST 

MDTP j 

Number sense 

✓ 


Algebra and functions 

✓ 


Measurement and geometry 3 

✓ 

✓ 

Statistics, data analysis, and probability 3 

✓ 

✓ 

Mathematical reasoning 

✓ 


Decimals, their operations and applications; percent 


✓ 

Simple equations and operations with literal symbols 


✓ 

Exponents and square roots; scientific notation 


✓ 

Fractions and their applications 


✓ 

Graphical representation 


✓ 

Integers, their operations and applications 


✓ 


CST is California Standards Test; MDTP is Mathematics Diagnostic Testing Project. 

a. A formal alignment study would be required to verify whether the CST and MDTP cover the same content 
standards in the topic. 

Source: Authors' analysis of primary data collected for the study. 






• How accurately did the combination of grade 6 math CST scale scores and grade 7 
MDTP topic scores predict proficiency on the algebra I CST in grade 8? How did 
this compare with the use of grade 7 math CST scale scores alone? 

• How consistent were predictions based on grade 6 math CST scale scores and grade 7 
MDTP test with predictions based on the grade 7 math CST scale scores alone? 

Analysis methods 

Research question 1. To examine the relationship between grade 6 math CST scores and 
proficiency on the algebra I CST in grade 8, the analysis focused on two subquestions. 

la. Were students who achieved proficiency on the grade 6 math CST in 2009/10 
also likely to achieve proficiency on the algebra I CST in grade 8 in 2011/12? 

The research team produced a two-by-two cross-tabulation of proficiency on these two 
assessments (proficient or not proficient on the grade 6 math CST by proficient or not 
proficient on the algebra I CST in grade 8). 

lb. To what extent did the grade 6 math CST scale score predict student proficien- 
cy on the algebra I CST in grade 8? 

• How were differences in grade 6 math CST scale scores associated with differences 
in the probability of achieving proficiency on the algebra I CST in grade 8? 

• What scale score would yield a 0.5 or higher probability of achieving proficiency 
on the algebra I CST in grade 8? 

• How accurately did grade 6 math CST scale scores predict proficiency on the 
algebra I CST in grade 8? 

The research team used a logistic regression to estimate the relationship between grade 6 
math CST scale scores (the predictor) and a binary outcome of proficiency on the algebra I 
CST in grade 8. 

The model takes the following form: 

Pr(Proficiency = 1) = logir'((3 0 + (3, CSX + £ 0j + s..), 

where CST.. is the grade 6 math CST scale score of student i in school district). (J 0 and are 
parameters estimated from the data presented as odds ratios, which identify how the odds 
of achieving proficiency on the algebra I CST in grade 8 change with a one-unit change in 
the independent variable. For example, indicates how the odds of achieving proficiency 
on the algebra I CST in grade 8 vary with a one-unit change in the grade 6 math CST scale 
score. l Qj is a district random effect, where ~ N( 0, Uj), and ^ i ■ represents the residual error 
term where £ ; . fc ~ N(0, 0). The logit function was used because the dependent variable is 
binary. This model is described more fully by Rabe-Hesketh and Skrondal (2012). 7 

The resulting estimates were used to plot the relationship between grade 6 math CST 
scale scores and the predicted probability of achieving proficiency on the algebra I CST in 
grade 8. This analysis addressed the subquestion on the association between differences in 
scale score and differences in the probability of proficiency and the subquestion on which 
scale score is associated with a 0.5 or higher probability of proficiency. 
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To assess how accurately grade 6 math CST scale scores predict proficiency on the algebra I 
CST in grade 8, the research team generated a cross-tabulation comparing predicted and 
actual proficiency outcomes using a 0.5 predicted probability as the cutoff for predicted 
success. Other probability cutoffs between 0.5 and 1.0 were also used for predicting success, 
to examine how changing the threshold for predicting proficiency affects the accuracy of 
the prediction (see tables B2 and B3 in appendix B). 

Research question 2. To examine the relationship between grade 7 MDTP results and 
proficiency on the algebra I CST in grade 8, three analyses were conducted based on three 
types of MDTP results: the number of MDTP topics mastered, the level of mastery of each 
topic, and the topic scores. 

2a. What was the relationship between the number of topics mastered on the 
grade 7 MDTP test and proficiency on the algebra I CST in grade 8? 

• Were students who mastered more topics more likely to achieve proficiency on the 
algebra I CST in grade 8? 

• If so, what was the minimum number of topics students needed to master on the 
grade 7 MDTP test to have a 0.5 or higher probability of achieving proficiency on 
the algebra I CST in grade 8? 

• How accurate was the prediction of proficiency based on the number of MDTP 
topics students mastered? 

The grade 7 MDTP test provides information on student mastery of seven topics. The 
research team produced a two-by-two cross-tabulation of the number of grade 7 MDTP 
topics mastered and proficiency on the algebra I CST in grade 8 (number of topics mas- 
tered on the grade 7 MDTP by proficient or not proficient on the algebra I CST in grade 8). 

The research team also analyzed the relationship between mastery of each MDTP topic 
and proficiency on the algebra I CST in grade 8, using a logit model. 

2b. What was the relationship between mastery of particular topics on the grade 7 
MDTP test and proficiency on the algebra I CST in grade 8? 

• How were differences in mastery of each topic associated with differences in the 
probability of proficiency on the algebra I CST in grade 8? 

• How accurately did a set of indicators of mastery of each topic predict proficiency 
on the algebra I CST in grade 8? 

The logit model takes the following form: 

Pr(Proficiency = 1) = login 1 ((3 0 + Xf^T.. + l 0j + 

where T .. is a dichotomous indicator for whether student i in school district j achieved 
mastery of MDTP topic area n and (J is the estimated difference in the odds of achieving 
proficiency on the algebra I CST in grade 8 associated with mastery of MDTP topic n 
(compared with nonmastery of that topic), with mastery of all other topics held constant. 

Once again, these estimates were used to assess how accurately the number of topics mas- 
tered on the grade 7 MDTP topics predicts proficiency on the algebra I CST in grade 8, 
generating a cross-tabulation comparing predicted and actual proficiency. A 0.5 predicted 
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probability was used as the initial cutoff for predicted proficiency. Various probability cutoffs 
between 0.5 and 1.0 were also used for predicting proficiency to examine how changing the 
threshold for predicting proficiency affects the accuracy of the prediction (see tables B2 
and B3 in appendix B). 

The final analysis for research question 2 focused on the association between topic scores 
on the grade 7 MDTP test and proficiency on the algebra I CST in grade 8. 

2c. To what extent did the grade 7 MDTP topic scores predict proficiency on the 
algebra I CST in grade 8? 

• How were differences in grade 7 MDTP topic scores associated with differences in 
the probability of achieving proficiency on the algebra I CST in grade 8? 

• How accurately did grade 7 MDTP topic scores predict proficiency on the algebra I 
CST in grade 8? 

This analysis is similar to the analysis for research question lb, except that the predictors 
were the continuous raw scores for the seven topics on the grade 7 MDTP test instead of 
the CST math scale scores. The model takes the following form: 

Pr(Proficiency = 1) = logirH^ + XPqfC. + l 0j + ej, 

where R .. is a continuous measure of the raw score on topic area n for student i in school 

mj r 

district j and [S is the estimated difference in the odds of achieving proficiency on the 
algebra I CST in grade 8 associated with a one-unit change in the raw score for MDTP 
topic n, with the scores on all other MDTP topics held constant. 

The research team assessed how accurately this model predicts proficiency on the algebra I 
CST in grade 8 using tabulations of actual proficiency rates among students with different 
predicted probabilities of achieving proficiency based on their MDTP scores. 

Research question 3. This question examined the use of a combination of grade 6 math 
CST scale scores and grade 7 MDTP topic scores to predict proficiency on the algebra I 
CST in grade 8 and examined the use of grade 7 math CST scale scores alone to predict 
proficiency: 

3a. How accurately did a combination of grade 6 math CST scale scores and 
grade 7 MDTP topic scores predict proficiency on the algebra I CST in grade 8? 

This analysis shed light on whether using grade 6 math CST scores and grade 7 MDTP topic 
scores together might improve algebra I placement decisions in grade 8. The research team 
estimated a logit model that used both grade 6 math CST scale scores and scores for each of the 
seven topics on the grade 7 MDTP test to predict proficiency on the algebra I CST in grade 8. 

The model takes the following form: 

Pr(Proficiency = 1) = logitr 1 ^ + ZP n R nij + P 9 CST ,j + l 0 j + hj)> 

where R . is a continuous measure of the raw score on topic n for student i in school dis- 
trict j. (J is the estimated difference in the odds of achieving proficiency on the algebra I 
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CST in grade 8 associated with a one-unit change in the raw score for MDTP topic n, 
holding grade 6 math CST scale score and scores on all other MDTP topics constant. (2 g is 
the difference in the odds of achieving proficiency on the algebra I CST in grade 8 associ- 
ated with a one-unit change in the grade 6 math CST scale score, holding the scores on all 
other MDTP topics constant. 

Model fit statistics (Tabachnick & Fidell, 2007) were used to examine whether this model 
provided a better fit to the data than either the model using grade 6 math CST scale scores 
alone (used in addressing research question lb) or the model using the topic scores of the 
grade 7 MDTP test alone (used in addressing research question 2c). Actual grade 8 profi- 
ciency rates among students were cross-tabulated with different predicted probabilities of 
achieving proficiency based on this model. The coefficients, standard errors, and statistical 
significance tests associated with this model were also calculated. 

3b. How accurately did the model based on grade 7 math CST scale scores alone 
predict proficiency on the algebra I CST in grade 8? 

To address this question, the research team estimated a logit model using grade 7 math 
CST scale scores to predict proficiency on the algebra I CST in grade 8. This model is 
identical to the model used to answer research question lb, except that it uses grade 7 
math CST scale scores in place of grade 6 math CST scale scores. To assess the accuracy 
of predictions based on this model, actual proficiency rates were cross-tabulated with dif- 
ferent predicted probabilities of achieving proficiency based on this model. 

Research question 4. To shed light on the potential effects of reassigning students to 
algebra I in grade 8 based on their grade 7 math CST scale score, this research question 
compares the consistency of predictions based on the combination of grade 6 math CST 
scale score and grade 7 MDTP topic scores with predictions based on grade 7 math CST 
scale score alone. 

4. To what extent were initial placement decisions based on the grade 6 math 
CST and the grade 7 MDTP test consistent with decisions based on the grade 7 
math CST alone? 

If using a combination of grade 6 math CST scale scores and grade 7 MDTP topic scores 
results in placement decisions that are similar to those based on grade 7 math CST scale 
scores alone, there would be no need to revisit spring placement decisions after grade 7 
math CST scale scores become available. 

To examine this question, the research team created a two-by-two cross-tabulation com- 
paring predicted proficiency on the algebra I CST in grade 8 (assuming a threshold of 0.5 
probability or higher) based on the model including a combination of grade 6 math CST 
scale scores and grade 7 MDTP topic scores and the model including the predicted profi- 
ciency based on grade 7 math CST scale scores alone. 

Finally, the research team also examined proficiency rates on the algebra I CST in grade 8 
among students who had discrepant predicted proficiency outcomes based on each model. 
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Limitations of the study design, data, and analyses 


First, the study sample was limited to the students from eight SVRA districts. Therefore, 
the findings might not be generalizable to students and districts outside the SVRA. 

Second, only the 2011/12 student cohort in grade 8 could be used in this study since this 
was the only cohort for which all the necessary data were available. SVRA districts started 
to use the MDTP tests districtwide only in 2011. 

This study does not provide evidence on the consequences of placing students who fall 
below certain passing thresholds into nonalgebra classes. Nor does it examine the course 
passing rates and academic trajectory of students who were not placed in algebra I in 
grade 8 because data are not available on what math courses students who scored below 
the placement threshold for algebra I took instead. 

Finally, this study focused on the test data used to assist with algebra I placement decisions 
in grade 8 because most of the SVRA district placement policies emphasize the use of 
assessment data for that purpose. Other criteria, such as end-oficourse grades or teacher 
recommendations, could be considered in a future study. The challenge of using such crite- 
ria is that the standards for assigning course grades and making recommendations may not 
be consistent across schools and districts. 
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Appendix B. Model comparisons 


This appendix summarizes the models used in the study, specifies which predictors were 
included in each model, and provides the resulting degrees of freedom along with the 
model fit statistics used to compare models (table Bl). Three commonly used model fit 
statistics are reported: log likelihood function, Akaike information criterion, and Bayesian 
information criterion. 

The log likelihood function is typically used to compare two nested models where there is 
at least one common predictor between models and one model is more complex than the 
other (includes additional predictors). For example, model 1 (in which the grade 6 math 
California Standards Test [CST] scale score is the predictor) is nested in model 4 (in which 
the grade 6 math CST scale score and grade 7 Mathematics Diagnostic Testing Project 
[MDTP] topic scores are the predictors). The Akaike information criterion and the Bayes- 
ian information criterion can be used to compare either nested or non-nested models. 
Because not all models in this study are nested in one another, these statistics were used 
to determine which model better fits the data. A model with a smaller Akaike information 
criterion and Bayesian information criterion is preferred. 

Because model 4 has the lowest values for these two statistics, it fits the data better than 
other models, although the difference between this model and model 5 is small. 

Another indication that model 4 outperforms the other models is that model 4 correctly 
classifies a slightly higher percentage of students (table B2). 

The accuracy of the model by various probability cutoffs is presented in table B3. Across 
various probability cutoffs, all models being compared appear to perform equally well in 
terms of accuracy. 


Table Bl. A model including the grade 6 math CST scale score and grade 7 MDTP 
topic scores (model 4) fits the data better than other models used in the study 

Model 

Predictor 

Log 

likelihood 

Degrees of 
freedom 3 

Akaike 

information 

criterion 

Bayesian 

information 

criterion 

1 

Grade 6 math CST scale score 

-1,302.34 

2 

2,608.68 

2,620.39 

2 

Grade 7 MDTP (mastery of seven topics) 

-1,316.74 

4 

2,641.48 

2,664.90 

3 

Grade 7 MDTP (seven topic scores) 

-1,247.39 

4 

2,502.77 

2,526.19 

4 

Grade 6 math CST scale score and 
grade 7 MDTP (seven topic scores) 

-1,180.48 

4 

2,368.96 

2,392.38 

5 

Grade 7 math CST scale score 

-1,195.14 

2 

2,394.29 

2,406.00 


CST is California Standards Test; MDTP is Mathematics Diagnostic Testing Project, 
a. Adjusted taking into account the nested nature of the data (students nested in districts). 
Source: Authors' analysis of primary data collected for the study. 
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Table B2. Across various probability cutoffs, the percentage of students correctly 
classified is the highest in a model including grade 6 math CST scale scores and 
grade 7 MDTP topic scores (model 4) 






Probability cutoff 



Model 

Predictor 

0.5 

0.6 

0.7 

0.8 

0.9 

1 

Grade 6 math CST scale score 

74 

74 

71 

67 

61 

2 

Grade 7 MDTP (number of topics mastered) 

76 

74 

72 

68 

62 

3 

Grade 7 MDTP (seven topic scores) 

77 

77 

75 

69 

61 

4 

Grade 6 math CST scale score and 
grade 7 MDTP (seven topic scores) 

78 

77 

76 

71 

65 

5 

Grade 7 math CST scale score 

78 

77 

73 

70 

64 


CST is California Standards Test; MDTP is Mathematics Diagnostic Testing Project. 
Source: Authors’ analysis of primary data collected for the study. 


Table B3. All models being compared performed equally well in terms of model 
accuracy 






Probability cutoff 



Model 

Predictor 

0.5 

0.6 

0.7 

0.8 

0.9 

1 

Grade 6 math CST scale score 

75 

80 

85 

92 

97 

2 

Grade 7 MDTP (number of topics mastered) 

77 

81 

84 

88 

93 

3 

Grade 7 MDTP (seven topic scores) 

76 

81 

86 

90 

94 

4 

Grade 6 math CST scale score and 
grade 7 MDTP (seven topic scores) 

77 

81 

87 

90 

97 

5 

Grade 7 math CST scale score 

78 

83 

84 

92 

96 


CST is California Standards Test; MDTP is Mathematics Diagnostic Testing Project. 
Source: Authors' analysis of primary data collected for the study. 
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Notes 


1. They are Alum Rock K-8 School District, Berryessa Union Elementary School Dis- 
trict, Franklin-McKinley Elementary School District, Milpitas Unified School Dis- 
trict, Mt. Pleasant Elementary School District, Oak Grove School District, San Jose 
Unified School District, and Sunnyvale School District. The first five school districts 
are among seven feeder school districts (the other two districts are Evergreen and 
Orchard) to East Side Union High School District. 

2. Students who are placed in a regular algebra course are expected to take the algebra I 
CST at the end of grade 8; students who are placed into a slower paced algebra course 
typically take the general math CST, which assesses achievement of California stan- 
dards in grades 6 and 7. 

3. Along with teacher recommendations, grades, and the other criteria applied across the 
districts. 

4. According to the developer’s website (http://mdtp.ucsd.edu, retrieved September 5, 
2013), the MDTP is currently field-testing new readiness test forms for the following 
CCSS courses: grade 7 math, grade 8 math, and algebra I. 

5. A sample report can be downloaded from http://mdtp.ucsd.edu/TestResults.shtml. 

6. Detailed test blueprints can be found on the California Department of Education 
website (http://www.cde.ca.gov/ta/tg/sr/blueprints.asp). 

7. Because the student-school relationship was not clearly identified in the longitudinal 
data received from the participating districts, the research team used district identifiers 
(instead of school identifiers) to take into account the nested nature of the data (stu- 
dents nested in districts). 
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The Regional Educational Laboratory Program produces 7 types of reports 



Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 


Stated Briefly 

Summaries of research findings for specific audiences 


Applied Research Methods 

Research methods for educational settings 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 



